Knowledge-based graphical interfaces for presenting technical information

Designing effective presentations of technical information is extremely difficult and time-consuming. Moreover, the combination of increasing task complexity and declining job skills makes the need for high-quality technical presentations especially urgent. We believe that this need can ultimately be met through the development of knowledge-based graphical interfaces that can design and present technical information. Since much material is most naturally communicated through pictures, our work has stressed the importance of well-designed graphics, concentrating on generating pictures and laying out displays containing them. We describe APEX, a testbed picture generation system that creates sequences of pictures that depict the performance of simple actions in a world of 3D objects. Our system supports rules for determining automatically the objects to be shown in a picture, the style and level of detail with which they should be rendered, the method by which the action itself should be indicated, and the picture's camera specification. We then describe work on GRIDS, an experimental display layout system that addresses some of the problems in designing displays containing these pictures, determining the position and size of the material to be presented

Multi-Language Edit-and-Continue for the Masses

We present an Edit-and-Continue implementation that allows regular source files to be treated like interactively updatable, compiled scripts, coupling the speed of compiled native machine code, with the ability to make changes without restarting. Our implementa-tion is based on the Microsoft .NET Framework and allows applications written in any .NET language to be dynamically updatable. Our solution works with the standard version of the Microsoft Common Language Runtime, and does not require a custom compiler or runtime. Because no application changes are needed, it is transparent to the application developer. The runtime overhead of our implementation is low enough to support updating real-time applications (e.g., interactive 3D graphics applications)

A Visual Language for Browsing, Undoing, and Redoing Graphical Interface Commands

We present the concept of an editable graphical history that allows the user to review and modify the actions performed with a graphical user interface. Using a pictorial metaphor borrowed from comic strips, an editable graphical history consists of a series of panels that depict in chronological order the important events in the history of a user's session. We discuss the visual language used in editable graphical histories, and describe Chimera, a graphical editor that generates these histories automatically. The user may scroll through the sequence of panels, reviewing actions at different levels of detail, and selectively undoing, modifying, and redoing previous actions. Chimera's editable graphical histories are constructed from parts of the editor window, the editor control panel, and the editor's pop up menus. Panels indicate both the objects that are modified and the actions performed on them. We describe the heuristics used to determine the objects depicted in each panel, the style in which they are drawn, and how actions are distributed among panels

Multi-Language Edit-and-Continue for the Masses

We present an Edit-and-Continue implementation that allows regular source files to be treated like interactively updatable, compiled scripts, coupling the speed of compiled native machine code, with the ability to make changes without restarting. Our implementa-tion is based on the Microsoft .NET Framework and allows applications written in any .NET language to be dynamically updatable. Our solution works with the standard version of the Microsoft Common Language Runtime, and does not require a custom compiler or runtime. Because no application changes are needed, it is transparent to the application developer. The runtime overhead of our implementation is low enough to support updating real-time applications (e.g., interactive 3D graphics applications)

Interactive Multimedia Explanation for Equipment Maintenance and Repair

COMET (COordinated Multimedia Explanation Testbed) is a research system that we are developing to explore the coordinated generation of multimedia explanations of equipment maintenance and repair procedures. The form and content of all material presented is generated interactively, with an emphasis on coordinating multiple media to allow cross-references between media and to make possible display layout that reflects the fine-grain relationships among the material presented. COMET's architecture includes multiple static and dynamic knowledge sources, a content planner, a media coordinator, media generators (currently text and graphics), and a media layout manager. Examples are given of the kinds of material processed and produced by each of the components

Coordinating Text and Graphics in Explanation Generation

To generate multimedia explanations, a system must be able to coordinate the use of different media in a single explanation. In this paper, we present an architecture that we have developed for COMET (COordinated Multimedia Explanation Testbed), a system that generates directions for equipment maintenance and repair, and we show how it addresses the coordination problem. In particular, we focus on the use of a single content planner that produces a common content description used by multiple media-specific generators, a media coordinator that makes a f'me-grained division of information between media, and bidirectional interaction between media-specific generators to allow influence across media.

Interaction and presentation techniques for shake menus in tangible augmented reality

Menus play an important role in both information presentation and system control. We explore the design space of shake menus, which are intended for use in tangible augmented reality. Shake menus are radial menus displayed centered on a physical object and activated by shaking that object. One important aspect of their design space is the coordinate system used to present menu op-tions. We conducted a within-subjects user study to compare the speed and efficacy of several alternative methods for presenting shake menus in augmented reality (world-referenced, display-referenced, and object-referenced), along with a baseline tech-nique (a linear menu on a clipboard). Our findings suggest trade-offs amongst speed, efficacy, and flexibility of interaction, and point towards the possible advantages of hybrid approaches that compose together transformations in different coordinate systems. We close by describing qualitative feedback from use and present several illustrative applications of the technique

Cross-Dimensional Gestural Interaction Techniques for Hybrid Immersive Environments

We present a set of interaction techniques for a hybrid user interface that integrates existing 2D and 3D visualization and interaction devices. Our approach is built around one- and two-handed gestures that support the seamless transition of data between co-located 2D and 3D contexts. Our testbed environment combines a 2D multi-user, multi-touch, projection surface with 3D head-tracked, see-through, head-worn displays and 3D tracked gloves to form a multi-display augmented reality. We also address some of the ways in which we can interact with private data in a collaborative, heterogeneous workspace